Transforming word vectors

In the previous week, I showed you how we can plot word vectors. Now, you will see how you can take a word vector and learn a mapping that will allow you to translate words by learning a "transformation matrix". Here is a visualization:

Note that the word "chat" in french means cat. You can learn that by taking the vector corresponding to "cat" in english, multiplying it by a matrix that you learn and then you can use cosine similarity between the output and all the french vectors. You should see that the closest result is the vector which corresponds to "chat".

Here is a visualization of that showing you the aligned vectors:

Note that $X$ corresponds to the matrix of english word vectors and $Y$ corresponds to the matrix of french word vectors. $R$ is the mapping matrix.

Steps required to learn $R$:

• Initialize R

• For loop

$Loss = \| XR-Y \|_F$

$g = \frac{d}{dR} Loss$

$R = R- \alpha*g$

Here is an example to show you how the frobenius norm works.

$\begin{array}{l}\mathrm{\parallel }\mathbf{X}\mathbf{R}-\mathbf{Y}{\mathrm{\parallel }}_F\\ \mathbf{A}=\left(\begin{array}{ll}2& 2\\ 2& 2\end{array}\right)\\ \parallel {\mathbf{A}}_F\parallel =\sqrt{2^2+2^2+2^2+2^2}\\ \parallel {\mathbf{A}}_F\parallel =4\\ \mathrm{\parallel }\mathbf{A}{\mathrm{\parallel }}_F\equiv \sqrt{{\sum }_{i=1}^m{\sum }_{j=1}^n{\mid a_{ij}\mid }^2}\end{array}$

\right) \\ \left\|\mathbf{A}_{F}\right\|=\sqrt{2^{2}+2^{2}+2^{2}+2^{2}} \\ \left\|\mathbf{A}_{F}\right\|=4 \\ \|\mathbf{A}\|_{F} \equiv \sqrt{\sum_{i=1}^{m} \sum_{j=1}^{n}\left|a_{i j}\right|^{2}}\end{array}

In summary you are making use of the following:

$\begin{array}{l}\bullet \mathrm{X}\mathrm{R}\approx \mathrm{Y}\\ \bullet \text{ minimize }\mathrm{\parallel }\mathrm{X}\mathrm{R}-\mathrm{Y}{\mathrm{\parallel }}_F^2\end{array}$